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MAGE MANAGEMENT SYSTEM 
FOR USE IN DERMATOLOGICAL EXAMINATIONS 

BACKGROUND OF THE INVENTION 
[0001] The present invention is a software-based system for use by 

physicians that want to use baseline photography to follow patients identified as having 
a high risk for developing cutaneous melanoma. 

[0002] In the United States there are currently nearly 5T,000 new cases 

of melanoma each year and more than 7000 melanoma deaths. If melanoma is 
1 0 identified early and removed promptly, many of these deaths can be avoided. 

[0003] Traditionally, melanoma identification was done using a method 

identified using the mnemonic ABCD. Physicians look for asymmetry, border 
irregularity, color variegation and diameter in moles on the surface of the patient's skin. 
However, not all melanoma can be detected early enough with the ABCD method. 
15 Melanomas may be difficult to discern and be buried in a background of other, benign 
moles. To put this later fact in perspective, it is estimated that there are 200,000 normal 
moles for every one melanoma. 

[0004] Focusing on patients who are at higher risk increases the odds of 

finding melanoma early. For example, it is estimated that 4% of the population of the 
20 United States will develop about 50% of the melanomas. Once this group is identified, 
annual follow up will pick up about 27,000 melanomas each year. 

[0005] Once melanomas are identified, they are excised immediately lest 

they start to metastasize. However, because of the 200,000 to one ratio of moles to 
melanomas and the cost of excision, it is important to not be too liberal in deciding that 
25 a mole is a melanoma. The total cost of excising suspicious-looking but non-malignant 
moles will quickly become very large with little appreciable health benefit. That fact 
underscores the importance of the answer to the question of how to determine if a mole 
is a melanoma early enough, before the ABCD method can be used. The "gray zone" is 
very large between, for example, a lesion that has unusual histopathologic features but 
30 lacking malignant potential and early melanomas. 

[0006] First, it is important to focus on those at higher risk for cutaneous 

melanoma. Fortunately, it is not difficult to identify patients with high risk. These are 
individuals who grow a relatively high numbers of dysplastic nevi; these are risk 
markers. Their existence marks the patient as someone who is at increased risk that one 
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of them will become malignant. However, once identified, these high-risk patients must 
be screened to find the melanomas, often among many, many moles and when the 
appearance of many of those moles may be suspect but not be so obvious as to be 
detectable using the ABCD method of the past. 

5 [0007] Thus there is a need for a better way to detect cutaneous 

melanoma, particularly on patients who are at elevated risk. 

SUMMARY OF THE INVENTION _ 
[0008] The present invention is a method and software-based system for 

using photographs to facilitate a dermatological examination of a patient's skin for 

10 cutaneous melanoma. The software permits high-speed viewing of images made from 
the photographs, because it is carried on the same compact disk as the digital images, 
and allows them to be seen at progressively higher resolutions and to be adjusted for 

brightness and contrast. 

[0009] The software is protected for privacy and security but otherwise 

1 5 launches upon being inserted into the compact disk drive of a general-purpose computer 
connected to a monitor. Initially, thumbnail images appear on the monitor of the 
computer. The images show portions of the skin of the patient. The user selects an 
image by clicking on it, which causes a medium-resolution version of the image selected 
to appear for a medium-resolution image of the area of the patient's skin shown in the 
20 thumbnail image. A higher resolution image can be obtained by marqueeing on a region 
of interest in the medium-resolution image. Using scrolling and marqueeing techniques, 
each region of interest can be examined and compared with images taken earlier in time. 
Images can also be adjusted for contrast and brightness. 

[0010] An important feature of the present invention is that one compact 

25 disk, that is easily storable in a patient's file, contains a complete, easily-usable and 
detailed record of the patient's skin that will allow a dermatologist to be better able to 
identify the presence of melanoma at an early stage, before the ABCD method could 
suggest that a dysplastic nevus might be a melanoma. The stand-alone record of the 
compact disk is not subject to loss of data as a result of computer failure. Also, the 
30 archival negatives and master disk can be preserved for replacement of the disk in the 
event it becomes damaged or misplaced. 

[0011] Another important feature of the present invention is the use of 

computer image management techniques to allow a user to navigate the images of a 
patient's skin. Moving from one image to the next is a simple matter of clicking on the 



WO 2005/060466 PCT/US2004/038044 

images to open and close them and to scroll in order to survey them quickly. The 
thumbnail images are presented in a rational order and one can move through them one- 
by-one or go immediately to the one of interest and, in a few actions familiar to users of 
conventional software applications, be viewing a particular mole at high-resolution. The 
5 use of computer image management techniques also allows the user to keep track of 
what portion of the patient's skin the user is viewing and provides a systematic approach 
to examination. This is particularly true when a patient has several seteof images done 
in a consistent way over time and especially when the patient has a large number of 
moles. The present image management techniques make it much easier to detect 
1 0 changes in the same mole on sequential photographs. 

[0012] Another important feature of the present invention is the use of 

both medium and high-resolution images. The medium-resolution images allow a 
dermatologist to scan a wider area and get an impression of the moles typical of a 
patient as a step preliminary to looking for unusual ones. The high-resolution images 
1 5 allow viewing the detailed structure of a mole' s appearance. 

[0013] Another important feature of the present invention is that the 

software application and the photos are on the same CD. This simplifies the viewing of 
the photos because no software needs to be installed on the computer in advance. Thus, 
any suitably equipped general purpose computer will be capable of permitting the 
20 images to be viewed, as long as it has a compact disk drive and one of a few widely 
. used operating systems, such as WINDOWS by Microsoft. This feature also makes it 
easier to compare subsequent sets of images with previous sets. Although the software 
may have evolved in the interval between sets, each set will have the appropriate 
software to permit viewing of the images on that compact disk. Thus, both sets will be 
25 viewable for comparison notwithstanding the fact that the software mat is used to view 
them may be different. 

[0014] Two other important advantages of the present invention are that 

melanomas can be detected earlier because historical records allow detection of small 
changes in small moles more easily and more objectively. Correspondingly, it is less 
30 likely that a mole will be excised merely out of an abundance of caution and a lack of 
evidence of its previous appearance. These two advantages combine to result in a 
higher rate of excised melanomas compared to excised non-melanomas and fewer 
unnecessary biopsies, which can each cost more than creating one set of photographic 
images for the present invention. 
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[0015] Other features and their advantages will become apparent to those 

skilled in dermatological examinations from a careful reading of the Detailed 
Description of Preferred Embodiments, accompanied by the following drawings. 
BRIEF DESCRIPTION OF THE DRAWINGS 
5 [0016] In the drawings, 

[0017] Fig. 1 is an illustration of the overview window, according to a 

preferred embodiment of the present invention, showing the first five_of 33 thumbnail 
images of a patient; 

[0018] Fig. 2 is an illustration of a window showing a medium-resolution 

1 0 image selected from one of the thumbnail images, according to a preferred embodiment 
of the present invention; 

[0019] Fig. 3 is an illustration of a window showing a high-resolution 

image of a marqueed portion of the medium-resolution image, according to a preferred 
embodiment of the present invention; 
15 [0020] Fig. 4 is an image of the control window showing the five 

adjustment controls for enhancing the high-resolution image, according to a preferred 
embodiment of the present invention; 

[0021] Fig. 5 illustrates the print dialog box for the high-resolution 

image shown in Fig. 4; 

20 [0022] Fig. 6 illustrates the save hi resolution image to disk dialog box for 

the high-resolution image of Fig. 4; 

[0023] Fig. 7 illustrates the annotation entry dialog box for the high 

resolution image of Fig. 4; 

[0024] Fig. 8 illustrates the survey sequence according to a preferred 

25 embodiment of the present invention; and 

[0025] Fig. 9 illustrates the survey sequence with acquired images according 
to a preferred embodiment of the present invention. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
[0026] The present method is a method for making and using an effective 
30 medical record of a patient's skin so that a dermatologist can methodically survey the 
skin in order to detect melanomas. Melanomas may be identified by altered rates of 
pigment production, altered types of pigment, altered growth rates, altered locations of 
growth and different clones that may survive better than other clones. In other words, 
although a mole may in the beginning look much like other moles, as time goes on, it 
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will declare itself by these characteristic changes in its appearance. Thus, it is important 
to have a clear, complete, historical record for use in comparing moles over time to 
detect the changes in order to identify melanomas early, before they metastasize. 

[0027] The present invention is useful in one part of a four-step process. 
5 That process is directed to early detection of melanomas. The first step is to inquire of 
the patient as to whether he or she is concerned about a particular mole. While a patient 
will likely not know if a mole is a melanoma or not, they often do pay attention to their 
bodies and their moles and will likely have noticed changes in them. These changes 
may be important and should be taken seriously. 
10 [0028] The second step of the process is a total body skin examination. 

Patients may tend to make moles of a pattern, and a mole present on the skin that 
departs from the pattern may be cause for concern. These have been called "ugly 
duckling" moles because they do not match the others. 

[0029] The third step in the four-step process involves dermoscopy. In step 
1 5 2, those "spots" that did not match the others can be identified but, through dermoscopy, 
a melanocyte nevus can be identified as seborrheic keratosis or thrombosed 
hemangioma or other benign lesion. 

[0030] The fourth step involves the use of a photographic record. A 
photographic record allows an assessment of changes in a mole. It is equivalent to a 
20 baseline EKG, chest X-ray, or set of laboratory results. Consistently made sets of 
photographs enable an objective comparison of moles over time. If a mole has changed, 

it is a candidate for excision. 

[0031] The extent of change must necessarily be somewhat arbitrary, 

depending on the length of time between images. 
25 [0032] Using tins process, it is likely that one in ten moles excised is a 

melanoma (or one in four for a patient that has severely atypical nevi and melanomas) 

but that is far better than removing 200,000 moles in order to remove the one melanoma. 
[0033] The present invention is thus a software-based application that allows 

the management of digitized images of a consistently taken set of photographs of a 
30 patent's skin for examination and comparison with a subsequently taken set The set of 

photographs includes the entire skin surface of the patient, preferably divided into areas 

for making a set of individual photographs, and most preferably approximately 33 

photographs. These areas of the patient include the following: 
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left side of head, neck and shoulders 
right side of head, neck and shoulders 
front of head, neck and upper chest 
back of head, neck and upper back 
middle of the back 
buttocks 
back of thighs 

backs of knees, calves and feet 
chest 

upper abdomen 
lower abdomen 



left torso, arm down 
left torso, arm up 
inside left arm 
right hip and thigh 
right lower leg and foot 
inside of right thigh 
inside lower right leg.aad foot 
left hip and thigh 
left lower leg and foot 
inside of left thigh 
inside of lower left leg and foot 
back of left and right forearms 
back of left and right hands 
front of left and right forearms 
both palms 
soles of both feet 



front of thighs 
front of knees, shins 
tops of both feet 
right torso arm down 
right torso arm up 
inside right arm 

[00341 Although there can be variations in the number and orientation of the 
photographic set, it is important that (1) the set of photographs must include the entire skin 
surface; (2) the set must include plural photographs so that much of the skin is photographed 
from a normal or near-normal angle; (3) the set should preferably be consistent with previous 
sets (angle, lighting, etc) obtained; and (4) the photographs in the set should be well-taken 
photographs, preferably professionally taken and developed with professional equipment and 
techniques to get the best images possible. The photographs are then digitized after 
development by high quality, professional methods as well and recorded on a compact disk with 
a copy of the image management software. The digitized photographs should have at least about 
six million pixels per image for the requisite resolution. 

[0035] The software allows high-speed management of the images for part of the 
examination of the patient. This means that the photographic images are organized and can be 
selected, compared, annotated, enlarged and adjusted easily and conveniently by the user so that 
the user can navigate through the patients' sets of skin images, examining easily and accurately 
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all moles of interest to detect those that have change and are therefore candidates for excision. 
Management also means that the images are preserved unaltered and left as organized as they 

were originally after examination. 

[0036] The following specifies all features and procedures of the present invention in 
further detail, winch invention is a software application for implementation on a general-purpose 
computer. The present invention is preferably a photographic display system on the read-only, 
writable, or re-writable memory of a compact disk (CD-ROM). It is designed to serve as an 
adjunct to the physical examination of a patient by a dermatologist or other specialist medical 
practitioner when following patients who are at high risk for developing cutaneous melanoma. It 
allows the rapid display of one or more sets of plural high-resolution color images of the patient's 
skin surface and permits efficient comparison of the patient's current condition with a set of base 
line images. 

[00371 The present software application provides several tools that aid in this 
comparison, including zooming, adjusting for brightness/contrast, high-resolution file output, 
printing of high-resolution images, annotating images, acquiring new images and associating sets 

of annotations and images. 

[00381 The present software application can be used to view and add images to a 
compact disk as a self-contained medical record with integral viewing software and that can. 
therefore travel with the patient should he or she move to another area or change physicians. As 
a self-contained medical record, the contents of the compact disk may be securely transmitted 
electronically to a remote system, copied to another memory device or media (CR-R, CD-RW, or 
DVD-R), and viewed on the remote system for consultation or comparison. 

[00391 The compact disk programmed with the present software application is a 
single executable file compatible with current, widely-used, operating systems, such as Microsoft 
Windows 98SE, Windows 2000, and Windows XP. Associated ancillary files reside on the 
compact disk volume and are necessary to the operation of the application, but are not integral 
elements of it. 

[00401 While most computing-related terms used herein possess connotations and 
denotations commonly understood in the industry, some terms employed herein have more 
specific and limited meanings and are defined below. 
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[0041] An "action" is any intervention by the user on an otherwise idle application 
state. An action always necessitates user interaction with one or more controls visible on the 
graphical user interface (GUI) or some key- or key-combination-presses. 

[00421 The "active window" is the "sole child window" if only one child window 
appeals in the parent window, or is the window currently enabled by the operating system to 
receive clicks, key-presses, or other user inputs. The active window always appears at the top of 
the Z-order in the graphical user interface (GUI) and is designated as the-active window by its 
active title bar color (or color gradient), whereas inactive windows display a grayed-out (inactive 
title bar color) title bar. When a control can initiate an action on several windows or types of 
windows but can affect only one window at a time, the active window is the only window that is 
affected by that action. 

[0043] The phrase "anchor annotation" refers to the baseline annotation that defines 
the starting point of a survey line and relates survey line dennoscopic annotations to the images 
against which the anchor annotation is made. 

[0044] "Annotation" refers to the recording of default and user-entered data in 
association with either data describing an image or a part of one, or to the image itself. 

[0045] "Baseline annotation" refers to an annotation made against a baseline image. 
[0046] A "baseline image" is a high-resolution image rendering of one of the 
medium-resolutions images on the compact disk. 

[0047] A "button" (or "speed button") is a control in the graphical user interface that 
is clicked on with the mouse pointer hovering over the depicted button image or similarly- 
affected by a hotkey or default key to initiate a user action. A phrase such as "a button closes a 
window" means that clicking the button causes the window to close. 

[0048] A "click" (or left-click, right-click, or double-click) is a user action that 
requires the pressing of a mouse button, touching of a peripheral input surface, pressing a 
keyboard key or key-combination, or other physical act of the user that generates an input to the 
application. 

[0049] The "client" area is the rectangular region inside the frame of the parent 
window. The title bar and its contents, the window frame, and any window menu are termed 
"non-client." 
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[0050] A "context menu" is a list of text menu options associated with a specific 
control. In the Windows common user access (CUA) GUI, the available context menu is 
presented upon a right-click action. 

[0051] A "control" is the smallest graphically and functionally cohesive element of 
the GUI that possesses meaning for the purpose of receiving input from or providing output to 
the user. 

[0052] "DD Locale" refers to a single, specifically-defined aspect, viewpoint, and 
area of a patient's body that is represented in one of the photographs of the set. 

[0053] Dermoscopic annotations are annotations made against a dermoscopic image. 

[0054] Dermoscopic image refers to an image of the human integument acquired by 
capture from a dermoscope or other small-field-of-view, close-up device specifically designed 
for the optical imaging of that subject and the recording of a particular set of its properties. 

[0055] "Drag" is the action of moving the pointer cursor while holding a 

simultaneous click action. 

[0056] A "marquee" is a transient rectangular line drawn while clicking and dragging 

across an image to select a region of the image on which to perform some operation. In use of 

the present software application, marqueeing is used to define the area of a medium-resolution 

window as a precursor to viewing a high-resolution window. 

[0057] "Selection" is an action on a control that indicates to the application upon 
which elements of the GUI default or to-be-chosen processing is to occur. Indication of the 
selection of a control or region thereof may be transient, such as the outline displayed in 
marqueeing, or may persist, such as in the display of a selected thumbnail image as a pressed 
button or a feature color change. 

[0058] The phrase "survey line" refers to a set of annotations defined by an anchor 
annotation and zero or more dermoscopic annotations made against images, all depicting the 
same dermal lesion or region of interest The purpose of a survey line is to store and identify 
annotated images to follow over time the appearance, condition, and evaluation of a lesion or 
region. 

[0059] The phrase "survey sequence" is the precisely-recorded and rigorously- 
preserved, partially chronological order in which a series of two or more annotations are 
displayed. Survey sequence is defined for the express purpose of facilitating the comparison of a 
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time sequence of two or more annotation images. A survey line in survey sequence is presented 
with its anchor image in the upper left corner of the visual display. To its immediate right is the 
chronologically earliest dermoscopic annotation. Immediately to the right of the earliest 
dermoscopic annotation is the latest (that is, the most recently captured) dermoscopic image. 
Following the latest image is the remainder of the dermoscopic annotations in strict 
chronological order. This sequence of images provides the physician with an image comparison 
display that facilitates careful comparison of the oldest and newest images, and therefore the 
highest contrast possible between any two of the dermoscopic annotation images on the survey 
line. 

[0060] The "Z-order" describes the location on the z-axis (height) of the Cartesian 
coordinate system that defines the location of any pixel or pixel set in the OS graphical display 
system. A window of a higher Z-order will obscure any part of a window of a lower Z-order that 
would otherwise appear 'behind' the higher. On a computer monitor, the Z-order decreases in 
the direction away from the user ("into the screen"). 

[0061] The present software application resides on a compact disk, from which it 
runs when invoked by an introduction program that is self-starting ("autoruns"), if enabled by the 
operating system, upon insertion of the disc into any CD-ROM drive. 

[0062] To preserve patient privacy and permit compliance with current health care 
privacy regulations, the present software application must be invoked with an "obfuscated run- 
time" parameter. Any attempt to execute the application without providing the parameter will 
result in immediate termination of execution without any image display. 

[0063] The application can be configured at the time of its writing onto' a CD-ROM 
or DVD to run only from a CD-ROM or DVD drive. An attempt to run the application from a 
hard drive or other type of storage medium would then result in immediate termination of 
program execution with no images ever having been visible. 

[0064] The software application reads only the image data resident on the disk, and 
cannot be used to access image or any other data anywhere else in the computing environment. 

[0065] All auxiliary files such as .DLLs and .OCXs, necessary specifically to the 
execution of the present software also reside on the disk. Therefore, the disk does not install any 
of its contents onto persistent storage in the user's computing environment in any manner. 
During normal operation, however, the operating system itself may cache or page-swap some or 
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all of the program executable or image data onto the local hard drive as such data may typically 
be transiently stored due to operating system resource limitations. 

[0066] Image data consist of standard medical photography that is internally stored in 
a proprietary format and optionally encrypted. The encryption algorithm implemented needs to 
be sufficient to ensure reasonable patient privacy and to provide adequate security to maintain 
compliance with current health care privacy regulations during passive storage of the disk, active 
viewing of the image content, or transmission of the complete image files between storage units, 
computer systems, or across networks or other dedicated, switched or connectionless 

communications links. 

[00671 During execution of the present software application, unencrypted image data 
is present in RAM (random access memory) and possibly in system swap files, and may remain 
in the swap files or on temporarily allocated swap file sectors after the application terminates. 
This unencrypted image data are not available except to entities with physical access to the 
machine hosting the application and are inaccessible to all other entities that are not authorized to 
have access to the hosting machine and local area network (LAN). 

[0068] The application implements a graphical user interface (GUI) that is modeled 
on common user access (CUA) specifications. The presentation layer of the user interface maps 
to a modified multiple document interface (MDI) model such as that found in operating systems 
such as Microsoft WINDOWS. In the MDI, a parent window is always visible in "full screen" 
mode by default, with perhaps one child window present in "minimized," "maximized," or. 
"partial" screen mode. 

[0069] The controls that generally cover the largest areas of the display are 
"windows," as that term is commonly understood in current user interface technology. Each is a 
movable, and sometimes resizable, rectangle typical of the supported operating systems 
prevalently used. Only one window at a time may hold the focus of attention of the application - 
namely, the "active" window. When any window-specific action occurs, the object of the action 
is the active window. Two classes of windows are implemented in the multiple document 
interface (MDI) of the present software application, the single "parent" window and perhaps one 
"child" window. The single parent window possible in the MDI appears upon execution of the 
present software application and is present during the entire lifetime of the executable. The 
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parent window opens and remains in full screen mode by default, and contains each child 
window that opens either by default or as a result of user action. 

[0070] Referring to Fig. 1 , at the top edge of the parent window, referred generally by 
reference number 10, a title bar 12 displays the name of the application executable and its 
version, and three window mode buttons: minimize 14, restore or maximize 16, and close 18. 
An overview window 22 appears in the parent window 10. Just below the title bar in the client 
region of the parent window, the tool bar 24 displays a set of speed buttons. Child windows are 
window controls that open within and are always visibly bounded by the client area perimeter of 
the single parent window. 

[0071] The overview window 22 contains one "thumbnail" view 26 of each image 
file resident on the compact disk. These "thumbnail views" are low-resolution views of the 
patient images 28 used to make larger-scale selections of the body locale to be viewed in greater 
detail. The data depicted with the image include tiie determination of the compact disk order 
source and destination, and the personal identifications of the patient, the physician and the 
photographer. 

[0072] To preserve patient modesty, only the first five images of the standard image 
set appear in overview window 22, as shown in Fig. 1, leaving the more sensitive images hidden 
from view. Use of standard scroll controls 30 allows the user to select a thumbnail image from 
the set of plural images. 

[0073] The overview window title bar 36 displays die standard icons common to 
operating systems such as WINDOWS by Microsoft, the window title, and three (3) enhanced 
window mode buttons 38: minimize, restore or maximize, and close. The minimize button 
reduces window size to a portion of its title bar and repositions it to the bottom left of the parent 
window. Double clicking on the minimized overview window restores it to the position and size 
displayed at the time of minimization. 

[0074] The overview window 22 can be enlarged to reveal additional images by 
choosing the maximize button at the top right of the overview window or by dragging the lower 
right corner until the desired set of images is revealed. Clicking the maximize button at the top 
right of the overview window 22 results in all thumbnail images becoming visible on the screen 
of the monitor. The lower right corner can be dragged until the desired set of images is revealed. 
Resizing of the overview window 22 always results in a window size and shape that displays 
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every visible image in its entirety without cropping. Once the maximize button is activated, it is 
replaced by the restore button. Clicking the close button closes the overview window 22. 

[0075] The overview window 22 does not provide a "context" menu. The overview 
window 22 does however provide one client control, the "thumbnail" images 26. When any 
thumbnail image 26 is clicked, a "medium-resolution" window 44 appears as the active window 
presenting the medium-resolution image 44 corresponding to the thumbnail image 26 selected, as 
illustrated in Fig. 2. The thumbnail image 26 in the overview window 22-4s highlighted in die 
same manner as a pressed virtual button until another thumbnail image 26 is selected Only one 
medium-resolution window 44 may be open for any given thumbnail image 26. When a 
thumbnail image 26 without any associated medium-resolution window 44 is clicked, a new 
medium-resolution window 44 opens. If the thumbnail image 26 for which a medium-resolution 
window 44 already open is clicked, instead of a new medium-resolution window opening, the 
associated medium-resolution window 44 becomes the active window. 

[0076] Using standard scrolling controls 30 and key presses common to operating 
systems such as WINDOWS by Microsoft, the thumbnail image 26 in the overview window 22 
may be scrolled. Scrolling proceeds one row of thumbnail images 26 at a time. The medium- 
resolution window 44 appears upon selection of a thumbnail image 26 in die overview window 
22. The image presented in the medium-resolutions window 44 can be used to delineate the area 
that a high-resolution window will present. Multiple medium-resolution windows derived from 
the same or different low-resolution (thumbnail) windows may be open at one time. 

[0077] The title bar 46 of a medium-resolution window 44 displays standard 
operating system menu icons, the window title 50 (that includes the number of the image in the 
overview window tiiat was selected to generate the medium-resolution window), and one 
window mode button 52, namely, "close." The close button 52 closes the active medium- 
resolution window 44. The medium-resolution context menu also presents "close" as the one 
action option. This close option also closes the active medium-resolution window 44. 

[0078] The medium-resolution window context menu presents five options: print, 
locate annotations, review annotations, save and close. The "print" option opens a standard print 
"dialog" box, typical of operating system such as WINDOWS by Microsoft. The options and 
capabilities available through this dialog box depend on the hosting computer setup, the version 
of the operating system, and the installed printer set The "locate annotations" option provides 
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the same functionality as the locate annotations speed button. The "review annotations" option 
opens the annotations review window. The "save" option opens a standard save dialog box. As 
with the print option, most options and capabilities available through this dialog box depend on 
the version of the operating system. The extension of the filename and the associated graphic 
file format of the file saved depend on the setting of the files of type "listbox" on this dialog box. 
Finally, the "close" option closes the associated high-resolution window. 

[0079] Selection of a medium-resolution window 44, by clicking on its client area, 
makes it the active window and consecutively brings any derived high-resolution windows 58 
(see Fig. 3) to the top of the Z-order in a sequence, which is determined by the operating system 
until the last window in the operating system list is at the top of the Z-order. Selection of a 
medium-resolution window 44 by clicking on its title bar 46 makes it the active window and 
brings it to the top of the Z-order, leaving derived high-resolution windows 58 in place. If a 
medium-resolution window 44 is closed, its derived high-resolution windows 58 remain 
displayed on the graphical user interface unchanged and unmoved. Medium-resolution windows 
44 may be moved within the client area of the parent window 10 by clicking on the window's 
title bar 46, dragging the window 44 to the desired position and dropping it there, which is a 
standard operating system interface action. 

[0080] A high-resolution window 58 provides a view of a selected area of the source 
medium-resolution image for comparison with the patient's current condition. A high-resolution 
window 58 opens upon completion of each selection of a region of interest performed on the 
active medium-resolution window 44 that delineates the area of the image that is to be presented 
in the high-resolution window 58. Selection of a region of interest is performed by 
"marqueeing" the medium-resolution image (selection of an area smaller than a default minimum 
during the marqueeing action will not result in high-resolution window being opened). Multiple 
high-resolution windows 58 derived from the same or different medium-resolution windows may 
be open at one time. 

[0081] The title bar 60 of the high-resolution window 58 displays a standard system 
menu icon. In particular, the window title bar 60 includes the number of the image 26 in the 
overview window 22 that was selected to generate the medium-resolution window 44 from 
which the high-resolution window 58 was derived, a single uppercase letter denoting the order in 
which the high-resolution window was derived from the medium-resolution window, the window 
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title, the magnification factor; and one window mode button, namely, a close button. The close 
button closes the active medium-resolution window. 

[0082] The high-resolution window context menu presents eight options: zoom in, 
zoom out, print, save, locate, annotate, drag/marquee, and close. The "print" option opens a 
standard print "dialog" box, typical of operating systems such as WINDOWS by Microsoft. The 
options and capabilities available through this dialog box depend on the hosting computer setup, 
the version of the operating system, and the installed printer set. The 'Jsave" option opens a 
standard save dialog box. As with the print option, most options and capabilities available 
through this dialog depend on the version of the operating system. The extension of the filename 
and the associated graphic file format of the file saved depend on the setting of the files of type 
"listbox" on this dialog box. Finally, the "close" option closes the associated high-resolution 
window. 

[0083] If the initial magnification of the high-resolution window is not adequate for 
evaluation or comparison purposes, the image may be viewed at a higher magnification factor by 
use of the zoom speed buttons or by marqueeing. The zoom speed buttons may be clicked to 
zoom in or out as desired to change the size of the image depicted and, therefore, its containing 
window by the selection of the magnification factor. When the zoom in speed button is used, the 
entire image expands by the pre-selected zoom step increment with each click until the image, 
can no longer fit within the parent window client area, whereupon scroll bars appear on the high- 
resolution window as needed to view portions of the image within the partially obscured entire 
image. The zoom in action can be repeated until the maximum magnification factor is reached. 

[00841 On the other hand, when the zoom out speed button is used on an image too 
large to fit within the parent window client area, the entire high-resolution image shrinks by the 
zoom step increment with each click until the image becomes visible whereupon any scroll bars 
disappear. Zoom out can continue until the pre-loaded minimum magnification factor-sized 
image is reached. 

[0085] When marqueeing is used to increase the magnification of the high-resolution 
window, the original extent of the image is replaced by the delineated region of interest, which is 
then magnified by a magnification factor increase of 200%. The zoom speed buttons may be 
used on a marqueed high-resolution window, but the loaded image cannot be fully recovered 
once it has been marqueed without reloading a new high-resolution window from a medium- 
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resolution window. The same maximum, minimum, and scroll bar controls are available for a 
marqueed high-resolution window as for a high-resolution window that has not been marqueed. 

[0086] In marquee mode, high-resolution windows may be moved within the parent 
window client area by clicking on the high-resolution title bar; dragging the window to the 
desired position, dropping the window there (as in standard operating system interface). 
Similarly, high-resolution windows may be moved by clicking on the high-resolution window 
title bar or client area, dragging the window to the desired position; and dropping it there. 

[0087] An image adjust window 74 is opened (see Fig. 4) by clicking the adjust 
image speed button when that button is enabled. The image adjust window is associated with the 
active window, which will be a high-resolution window, at the time of clicking. The image- 
adjust window permits real-time adjustment of the intensity and brightness of the associated 
high-resolution window. 

[0088] When the "image adjust" window 74 is open by clicking, it is associated with 
one particular high-resolution window 58 when clicked. If a different high-resolution window 
58 becomes the active window, any pending image adjustments are abandoned, and subsequent 
adjustments are applied to the current high-resolution window 5 8. 

[0089] As adjustments are made, the image in the associated high-resolution window 
visibly changes to reflect these adjustments. On some hosting machines,' visible image changes 
may lag behind Control actuation, dependent on the displayed size of the high-resolution window 
and the capabilities of the hardware and software extant on the hosting machine. The title bar 76 
of the image adjust window 74 displays a window title that includes the name of the window - 
'Adjust', the number of the image in the overview window 22 that was selected to generate the 
medium-resolution window 44 from which the high-resolution window 58 was derived that will 
be adjusted by the image adjust window 74, a single uppercase letter denoting the order in which 
the high-resolution window 58 was derived from the medium-resolution window 44; and a close 
window mode button. The close button closes the image-adjust window 74 without permanently 
applying any displayed adjustments to the associated high-resolution window. Also, the image- 
adjust window has no context menu. 

[0090] The image-adjust window 74 provides five controls for the adjustment of 
images. These controls include three buttons: reset 82, cancel 84, and "OK" 86, and two sliding 
controls 88, 90: one for intensity and the other for contrast. The reset button 82 reloads the 
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associated high-resolution window 58 in its unadjusted state as stored on the compact disk or as 
it was accepted if any previous adjustment action that has occurred during the current execution 
of the present software application. The cancel button 84 closes the image-adjust window 74 
without permanently applying any of the displayed adjustments to the image. The OK button 86 
applies the displayed adjustments to the associated high-resolution window 58 and closes the 

image-adjust window 74. 

[0091] The intensity sliding control 88 adjusts the intensity level (overall brightness) 
of the image in the associated high-resolution window 58. This sliding control allows 
adjustment within an absolute range that ensures that a usable image results. The sliding control 
displays adjustments relative to the intensity level of the unadjusted image over a range of - 
100% to +100%. 

[0092] The contrast sliding control 90 operates is a similar fashion to adjust the 
contrast level (light versus dark spread) of the image in the associated high-resolution window 
58. This sliding control aUows adjustment of contrast" within an absolute range that ensures that 
a usable image results and displays adjustments relative to the contrast level of the unadjusted 
image over a range of -100% to +100%. 

[0093] The "print hi-resolution image" window 94 (see Fig. 5) opens after clicking 
the "print high-resolution image" speed button 98. The window 94 opens as a print dialog box. 
The controls and functionality available through this dialog depend on the operating system 
version, installed printer set, and printer driver versions extant on the hosting machine. The print 
dialog is a system modal dialog and no other control may be actuated within the application until 
this dialog is completed. The print high-resolution image window 94 must be closed before any 
other window can become the active window or any button or control not available within the 
dialog window can initiate any action. 

[0094] The "save high-resolution image" window 100 opens after clicking the save 
high-resolution image speed button 102 (see Fig. 6). The window 100 opens as a standard "file 
save" dialog box. The controls and functionality available through this dialog depend on the 
operating system version and other hosting machine-specific conditions. The file save dialog 
box is a system modal dialog box and no other control may be actuated within the application 
until this dialog is complete. The save high-resolution image window 100 must be closed before 
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any other window can become the active window or any button or control not available within 
the dialog window can initiate any action. 

[0095] The saved file name will receive the file extension displayed in the file save 
dialog box 104 from among the types of file extension in the "listbox" 106 and will be stored in 
the graphic file format associated with that extension. 

[00961 The "annotation entry" window 1 16 opens when the enabled "add annotation" 
speed button 118 is clicked. When the "acquire" speed button 120 is down (active), and either 
an image acquisition device reports that an image has been acquired or an image is dropped on a 
baseline panel in the open annotation review window, the annotation entry window title bar 1 18 
includes the name of the window "annotate" and the image identification trailer contents from 
the title bar of the high-resolution window containing the image to be annotated. The title bar 
118 also shows one window mode button, namely, a close button 119. The close button 119 
closes the annotation entry window 116 without recording any annotation data that may have 
been entered for the image displayed. 

[0097] The annotation entry window 1 16 shown in Fig. 7 provides nine controls for 
the annotation of images, although no context menu. These controls include three speed buttons, 
four display boxes, and two radio button groups. The three speed buttons includes a reset, a 
cancel and an OK button 122, 124, 126, respectively. The reset button 122 clears data entered in 
the optional annotation entry controls and resets all controls with defined defaults to those 
default values. The cancel 124 button closes the annotation entry window without storing any 
annotation data for the image. The OK button 126 accepts default and entered data, writes the 
annotation data to a file on the compact disk, and closes the annotation entry window 116. 

[0098] The four display boxes are for the display of the current date, time, "survey" 
and "capture." The survey box displays the survey line of images into which the annotation data 
will be entered. The capture box 130 displays the method of capture that acquired the image 
being annotated, for example, a baseline image acquired during a total body photography session 
or a dermoscopy image acquired during a dermoscopy session. 

[0099] The radio button groups include a source button 132 and a type button 134. 
The source button 132 is for identifying the user entering the annotation from the list (physician, 
patient, and "see notes"). Unless in guided self-examination procedure mode (to be described in 
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more detail below), the default value of the "source" is "physician" for each annotation entered. 
When in self-examination mode, the default type of source is "patient." 

[00100] The second radio button includes the type or reason for entering the 
annotation from the following list: patient concern, survey, existing, negative, positive, no 
change and "see notes." When not in self-examination procedure mode, the default type is 
"survey." When in self-examination procedure mode, the default type is "patient concern." 

[00101] A "single line text entry" box 136 is provided for a shott identifier, that is, a 
comment. A "multi-line text entry" box 138 is provided for more extensive information entry 
such as the physician's notes. An image box 140 displays a copy of the image under annotation 
so that the annotation data is visually associated with the image. 

[00102] The "annotation entry" window 1 1 6 also permits entry of a non-default source 
and type by changing the selection in the corresponding radio button group. 

[00103] The present system has an "annotation review window" 148 (see Fig. 8) that, 
when opened, displays one or more baseline panels 150 that contain all annotated baseline 
images, each positioned adjacent to its related annotation data. Each baseline panel comprises a 
baseline image, a survey line, and the date, time, capture method, comment, source, type of 
annotation, and the annotation notes. The baseline panels with their annotations are displayed in 
rows arranged in chronological order and from left to right, unless otherwise specified by the 
user. As each row reaches the right edge of the client area, a new row is begun below the 
previous one at the left edge of the client area. Scroll bars automatically appear and disappear in 
the annotation review window as needed when its size changes in relation to the pixel 
dimensions of the client area. 

[00104] When any one baseline panel 150 is selected, all other base panels are grayed 
to indicate they are inactive, and a set of one or more survey series-related dermoscopic panels 
are displayed adjacent to the active baseline panels in chronological order of the survey. Each 
dermoscopic panel has nine display controls. These are a dermoscopic image, a survey line 
index, date, time, capture method, the comment, the source, the type of annotation and any 
annotation notes. Any pair of dermoscopic panels can be selected for side-by-side comparison, 
as illustrated in Fig. 9 by clicking on the two desired panels 150' and 150" in succession. As 
each is clicked, the dermoscopic panels are highlighted, as if they were pressed buttons. Once 
two panels 150', 150", have been selected for side-by-side comparison, the later in chronological 
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sequence will be moved in adjacent relationship (side-by-side) to the first in sequence so that 
time- varying conditions can be detected and evaluated by the user. 

[001051 After a side-by-side comparison, selection of a third panel will deselect both 
of the previously selected panels 150', 150", and return the moved panels to their original slots in 
the default, chronological sequence. It will also set the just-selected panel as the first panel in a 
pair of panels to undergo side-by-side comparison. Selection of any baseline panel during a 
comparison of dermoscopic panels, including the baseline panel for the currently compared 
dermoscopic panels, will deselect all previously selected panels and return each to its default 
position. 

[001061 Two dermoscopic panels may be selected as candidates for overlay 
comparison (not shown). To overlay the selected dermoscopic panels, the first selected panel is 
dragged over the second panel. The two images are separately filtered to permit an enhanced 
display of the combined panels in which the images are digitally combined into one. The 
annotation data display is modified by blanking the annotation data that may conflict between 
these dermoscopic panels and by displaying the annotation date and time pair of each panel 
adjacent to each other so that the annotation sequence is clear. A single copy of the dermoscopic 
image and survey data common to each is also displayed. 

[001071 Once the dermoscopic images are combined and displayed, the image 
chronologically later in time may be rotated and translated to fit it as closely as possible over the. 
earlier image by clicking in certain image areas. For example, clicking in the center of the image 
area will set the mouse cursor to the drag graphic, will allow real time orthogonal X- and Y- axis 
translation (by dragging) of the later image over the earlier image. The two images will be 
recombined when the dragged image is dropped in position over the earlier image. By clicking 
on any point toward the edge of the image area outside the center half, the mouse cursor will be 
set to the rotate graphic, which allows real time rotation (by dragging) of the later image around 
its center over the earlier image. The two images will be recombined once the dragged image is 
dropped into position. 

[001081 As 311 enhancement to the common user access specification intended to 
increase simplicity of the user interface, no text-based menu appears at the top of the parent 
window 10. Instead only a tool bar 24 displaying speed buttons containing graphic icons appears 
in the parent window 10. Each speed button is enabled or disabled (shown as grayed-out) 
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depending on content and states of child windows. Hovering the pointer cursor over each speed 
button, whether enabled or not, will display a pop-up tool tip providing a brief description of the 
purpose of the button. 

[00109] The "load image files into overview" speed button 156 closes all open 
medium-resolution windows 44 and high-resolution windows 58, then initiates a loading or re- 
loading of the low resolution images 26 from the compact disk into memory, culminating in their 
display in the overview window 22 at its default size of five (5) images from those loaded, one 
each for the first five dermoscopic display locations. The "display overview" speed button 156 
restores the overview window 22 if in another mode or not in view and moves the overview 
window 22 to the top of the Z-order. 

[00110] Referring to Fig. 1, the "zoom in" speed button 158 increases the 
magnification factor of the active high-resolution window 58 by the percentage set in the select 
zoom speed button. The high-resolution window 58 opens at 400% magnification factor (or 
400% magnification of the medium-resolution window) and can be increased by the pre-selected 
zoom step size up to a maximum 3200% magnification factor. A single click applies a single 
step of magnification factor increase; sustaining the click will result in auto-repeat of the increase 
increases. This speed button 158 is enabled whenever one or more high-resolution windows 58 is 
visible in the parent window 10 and disabled otherwise. 

[00111] The "zoom out" speed button 160 decreases the magnification factor of the 
active high-resolution window 58 by the percentage set in the pre-selected zoom speed button. 
The magnification factor of the high-resolution window 58 can be decreased from any previous 
increase above the loaded multiplication factor down to the default loaded (minimum) 
magnification factor of 400% (400% of medium-resolution factor). A single click applies a 
single step of multiplication factor decrease; sustaining the click will result in auto-repeat of the 
decrease. This speed button 160 is enabled whenever one or more high-resolution windows 58 
are visible in the parent window 10 and disabled otherwise. 

[00112] The "select zoom step size" speed button 162 presents options to select a 
zoom step size from 10, 20, 33, 50, and 75%. This speed button is enabled whenever one or 
more high-resolutions windows 58 are visible in the parent window 10 and disabled otherwise. 
The option setting of this speed button 162 persists for the duration of any session until the user 
causes it to change. 
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[001131 The "zoom reset" speed button 164 resets the active high-resolution window 
58 to the default loaded magnification factor. The reset button 164 resizes the active high- 
resolution window 58 to the default loaded magnification factor. If the high-resolution window 
58 has been marqueed, the reset image will display only the region of interest selected during the 
last marqueeing action. This speed button 164 is enabled whenever the active window is a high- 
resolution window 58 and it is displaying at greater than the load default 400% magnification 
factor. 

[00114] The "marquee/drag high-resolution image" speed button 166 can be toggled 
between two modes of action on high-resolution windows 58. This speed button 166 is enabled 
whenever one ore more high-resolution windows 58 are visible in the parent window 10 and 
disabled otherwise. The default state of the marquee/drag high-resolution image speed button 
166 is marquee, the non-default drag state persists for the duration of the session or until changed 
by the user. The two toggle modes are: to marquee the active high-resolution window 58 with 
the mouse to zoom in on the regions of interest and to drag the active high-resolution window 58 
by clicking on either its title bar 60 or client area to move the window 58 on the monitor screen 
in the parent window 10. 

[00115] The "adjust image" speed button 168 (contrast/brightness) presents a dialog 
window that permits the adjustment of the contrast and or the brightness of the active high- 
resolution window 58 to aid in the detection of fine optical details of interest to the user. This 
speed button is enabled whenever one or more high-resolution windows 58 are visible in the 
parent window 10 and disabled otherwise. 

[00116] The "locate high-resolution image" speed button 170 opens the source 
medium-resolution window 44 if not already open and draws a gray rectangle around the region 
on the medium-resolution image that corresponds to the displayed high-resolution image. It 
promotes the medium-resolution window 44 to the top of the Z-order. The location rectangle 
will persist on the medium-resolution window 44 while the derived, located high-resolution 
window 58 is visible until the latter is closed. This speed button 170 is enabled whenever one or 
more high-resolution windows 58 are visible in the parent window 10 and is disabled otherwise. 

[00117] The "print high-resolution image" speed button 98 opens an operating system- 
standard print dialog window to select the printer to which and the properties and configurations 
values with which the active image is to be output. This speed button 98 is enabled whenever 
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one or more high-resolution windows 58 are visible in the parent window 10 but is disabled 
otherwise. The save high-resolution image speed button 102 opens an operating system-standard 
save file dialog window to select the location and name of the image file to be saved to the 
chosen storage medium. This speed button 102 is enabled whenever one or more high-resolution 
windows 58 are visible but is disabled otherwise, 
v [00118] The "privacy" speed button 172 provides a very rapid method of ensuring 

patient privacy by instantly hiding all images with the click of one buttoa-or restoring them to 
view if previously hidden. The privacy action may also be initiated by using a "hot key," 
specifically the keyboard combination "Alt-P." While accidental disclosure of patient health 
' information is not strictly forbidden by government regulations, patients are becoming more 
aware of their perceived and mandated rights of privacy, so this control offers a quality of 
experience to the patient that goes " speed button 174 toggles the beyond the required minimum 

[00119] The "self-examination" guided self-examination. This is a major feature of 
the present invention. It provides a standardized, assisted, prompted, self-paced procedure for 
patient self-examination. A script is compiled into the application-executable file as a text 
resource to define this procedure. It guides the patient through a specially ordered sequence of 
the dermoscopic display locales based on standard procedures for self-examination of the whole 
body. According to this procedure, the steps are subdivided into a manageable collection of 
smaller areas of the skin for examination by iterating through the entire set of medium-resolution, 
images by sequentially displaying each image, pausing at each to allow the patient to view the 
selectable magnification specific regions of interest on the image, referring to the help file for the 
self-examination procedure instructions and additional guidance as needed, optionally annotating 
regions for future reference or presentation of the annotated views to a physician for 
consultation, and optionally printing out the previous locales for review or further examination. 

[00120] The "self-examination" speed button 174 is in the up position at startup. 
When enabled and clicked, the self-examination mode is activated, the speed button 174 displays 
itself in the down position and remains there until it is clicked again or a next or previous action 
iterates past either end of the self-examination procedure image sequence. The self-examination 
procedure speed button 174 is enabled whenever one or more medium-resolution windows 58 
are visible in the parent window and disabled otherwise. 
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[00121] The "previous" speed button 176 closes any open medium-resolution 
windows and opens a new one displaying the previous DD Locale in the guided self-examination 
procedure sequence. If the open medium-resolution window 44 displays the first locale in the 
self-examination sequence, then the self-examination mode is exited. This speed button 176 is 
enabled whenever the system is in the self-examination mode but is disabled otherwise. 

[00122] The "next" speed button 178 closes any open medium-resolution windows 44 
and opens a new one displaying the next DD Locale in the self-examination procedure sequence. 
If the open medium-resolution window 44 displayed is the final locale in the image sequence, the 
self-examination mode is exited. This speed button 178 is enabled whenever the system is in the 
self-examination mode but disabled otherwise. 

[00123] The "mirror" speed button 180 is a toggle reverses function that displays the 
image in any medium-resolution or high-resolution window as if viewed in a mirror. To indicate 
that the image is reversed, it is displayed with a surrounding border having a bright color. This 
speed button is enabled whenever the system is in self-examination mode but disabled otherwise. 

[00124] The "locate annotation" speed button 182 is a toggle that enables or disables 
the application to display a colored rectangle on each medium-resolution window 44 for each 
annotation that exists for that window. This speed button 1 82 is enabled only when one or more 
annotated medium-resolution windows 44 are visible. 

[00125] An "add annotation" speed button 1 84 opens the annotation entry window 1 1 6> 
to permit entry of optional annotation data and change from the default values of annotation data, 
if desired. This speed button 184 is enabled only when one or more high-resolution windows 58 
are visible in the parent window 10. If an annotation is entered while the application is in self- 
examination mode, the "Source" defaults to "patient." Otherwise the default is "physician." 

[00126] The "review annotation" speed button 186 is a toggle that enables or disables 
the application to permit review of all annotations for medium-resolution windows 44 visible in 
the parent window 10. This speed button 186 switches to down when activated and die 
annotation review window is opened and is only enabled when one or more annotated medium- 
resolution windows is visible in the parent window. 

[00127] The "acquire" speed button 120 is another toggle that is enabled to allow the 
system to accept the capture and storage of an image from the default compatible image storage 
or acquisition device. When enabled, the "acquire" speed button 120 is down (activated), the 
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application will either (1) respond to an image captured message from any compatible image 
capture device by opening an annotation entry window 116; (2) accept an image dragged from 
another concurrently executing program and dropped onto a baseline panel 150 in the open 
annotation review window 148 to set default properties (such as annotation date, survey line, 
etc), and add the image to the survey line image set; or (3) respond to an attempt to drop a 
dragged image anywhere on the application graphical user interface other than a baseline panel 
1 50 by prompting the user to open the annotation review window 148 so that any extant baseline 
annotations are available to accept the image of which dropping was attempted, and by 
prompting the user to create a new or select an extant baseline annotation to accept the image of 
which dropping was attempted. If the "acquire" speed button 120 is in the up position, messages 
from any capture device are ignored and the application refuses to accept images dragged and 
dropped onto the graphical user interface. 

[00128] At application startup, the hosting computer is scanned for an available 
compatible dermoscopic image storage or acquisition device. If found, the "acquire" button 120 
is enabled. If not found, the "acquire" button 120 is enabled only in response to a "connection" 
message. If a compatible dermoscopic image storage or acquisition device is connected to the 
host computer while the application is executing, the "acquire" button 120 will be enabled in 
response to the connection message. 

[001291 The "help" speed button 188 displays the first page of information, 
instructions, diagrams, screen shots, or other text or visual data that assists the, user in the 
operation of the application and in understanding the contents of the screen as displayed. 

[001301 It will be apparent to those skilled in the art of dermatological examinations 
that many modifications and substitutions can be made in the foregoing description of preferred 
embodiments without departing from the spirit and scope of the present invention, defined by the 
appended claims. 
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